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Introduction 




Few scientific investigators can maintain expertise in statistical 
methods. They may have had substantial '•.raining in, understanding of, 
and competence in statistical methods; but they are unlikely to exercise 
these skills often enough to maintain them at a high level of proficiency. 

These investigators can use and are typically receptive to guidance in 
their statistical work provided they remain in control of their own analyses. 
Investigators with lesser statistical skills will benefit from even more 
directive guidance through the maze of detail required of good statistical 
practice. For all investigators, the tedium of computation or alternatively, 
thp maintenance of esoteric computer expertise, is a regrettable hindrance - 
to their function of extracting meaning from data. 

The system described here is an interactive computer-based system for 
assisting investigators on a step-by-step basis in the use of a particular 
analytic tool — Bayesian analysis using the two parameter normal model. 

The example is meant to be suggestive of the kinds of computer-assisted 
data analysis programs that can be developed for use by scientific investi- 
gators. Programs such as these can also be used in the classroom and laboratory 
for teaching purposes, but beyond this, they can be used by the practicing 
scientist in his day-to-day work. 
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The present program begins by assisting the investigator in the 
specification of his prior distribution of the unknown standard deviation 
and the unknown mean as a particular member of an appropriate extended 
natural conjugate class. As defined here, this class is somewhat broader 
than usually defined and, hence, more likely to accurately reflect prior 
knowledge or beliefs. For each parameter, the investigator is asked to 
specify three pieces of information about his prior information while, in 
fact, only two of these are necessary to determine this distribution. The 
requirement for three pieces of information typically evokes an inconsistency 
on the part of the investigator. The resolution of this inconsistency through 
sequential adjustment of judgments results in the statement of a coherent 
prior distribution. Throughout the interrogation, the investigator remains 
in control of his specification of prior information and is given repeated 
opportunity to revise his judgments, explore further, and reject simplifying 
statements. 

An important feature of this program is that it interacts with the 
investigator in the English language. The investigator need not be familiar 
with computer languages or with the internal workings of the computer. He 
•leed only learn how to sign-Ln and sign-off the terminal and to make simple 
alphabetic and numeric responses. 

Some investigators, however, will have a somewhat more fully developed 
understanding of the workings of the interactive program. For these investigators, 
the interactive program provides some techniques for speeding up the analysis. 

An investigator who has; used a program a number of times will probably feel 
that he needs a decreasing amount of guidance. Built-in "interrupt" and 
"go to" functions make it possible for him to proceed more quickly as his skills 
increase. For example, the investigator who is sure of his prior distribution j 

i 

can go directly to the posterior analysis. 
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The program also makes it possible for the investigator to consider 
easily a host of subsidiary questions that might be raised. He can, for 
example, get some idea of whether his contemplated sample size is adequate 
by pretending that he has obtained certain results and then seeing whether 
or not the posterior distribution given these fictitious data is tight 
enough for his needs. He can also similarly determine what sample size 
would be required to sufficiently modify the beliefs of a person holding 
different prior beliefs than his. A typical result of this kind of analysis 
is the realization that rather large samples are usually required to modify 
some prior beliefs — samples that may be larger than are economically feasible 
to obtain. 

We would emphasize that the program described here, while useful in its 
present form, should be thought of primarily as the prototype of more detailed, 
sophisticated, and complete programs. Our purpose in distributing this 
preliminary report is to encourage other investigators to become interested 
in some of the problems that evolve from this approach to interactive computing. 
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The Problem 

We shall illustrate the use of the CADA program for a specific 
problem that .is typical of a type of problem that arises with high frequency. 

An investigator needs to get a pretty good idea of the level of attainment 
of college-bound graduating high school students in the state of Iowa. He 
needs this information quickly. To provide some information, he looks 
at some data obtained from a published report with the hope that he will 
not have to get further data. 

The information he has consists of scores on The American College Test 
for twenty students who participated in a statewide survey. He has examined 
the process by which these students were selected and finds no evidence to 
challenge the assertion that this is a random sample of current college- 
bound students. More specifically, he has no information to distinguish 
these students from other students in the population. 

The background information available to him is as follows; He knows that 
nationally the mean ACT score is 20, and the standard deviation is 5 . He 
knows that the state of Iowa has a very high literacy rate and a very high 
per pupil expenditure on schools. This leads him to expect a mean value 
greater than 20. Also, as a statistician, he expects, on the average, a 
greater homogeneity in one state than across states. 

Undoubtedly, some investigators will have more specific prior information, 
and thus, we must expect that different investigators will wish to use 
different prior distributions. Our task, is to assist each investigator 
in quantifying his prior beliefs. We shall follow our investigator through 
the CADA program, annotating the ouput where some explanation seems useful. 

We shall then follow a second and, substantively better informed investigator 
through the same program, finally, we investigate the size of sample that might 
be required to get consensus from the two investigators. 
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standard deviation of ( 1.214). The 50% credibility interval about the mean is from ( 20.1815) to ( 21.8185) 



marginal distribution on the population mean is a t distribution on ( 44.0) degrees 




The posterior marginal density on the population standard deviation is an inverse Chi distribution 
on ( 40.0) degrees of freedom with scale factor ( 32.0986). The posterior marginal mode 

is ( 5.0130) , the posterior median is ( 5.1181), and the posterior mean is ( 5.1732) 

The 50% credibility interval is from ( 4.6540) to ( 5.4205) 
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Now considering your prior information about the mean and assuming that your modal 
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Your prior conditional distribution for the mean (given that the true standard deviation is 5.3000) 
is Normal, with a mean of ( 21.5000), a variance of ( 1.4045), and a standard deviation of ( 

The 50X credibility interval around the nean extends from ( 20.7006) to ( 22.2994). 
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The joint posterior for the mean and the standard deviation is ( 21.3889, 5.1883) 

This is the point on the plane around which the probability is most highly concentrated. 
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The posterior marginal distribution on the population mean is a t distribution on ( 69.0) degrees 
of freedom. It has a mean, median# and mode of ( 21.3571)^ and a standard deviation of ( 

The 50 % credibility interval extends from { 20.9302) to ( 21.7841). 
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The Interactive System for Bayesian Analysis of Educational Data that you have been using was 
developed under the sponsorship of the Iowa Testing Program (ITP) , and the American College Testing 
Program (ACT) , with the support of the University of Iowa Computer Center (UCC). 
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The exploration described in this example can now be done at the 
University of Iowa Computer Center in less than 30 minutes and at a cost of 
less than four dollars. Further economies will be forthcoming when the program 
scripts are transferred from slow core to disk storage. Even so, with the 
present technology, costs will remain sufficiently high to preclude use of 
the system on a routine basis. 

The present technology is somewhat unsatisfactory in another way. 

Sitting at the terminal, an investigator with even very limited exposure to 
the system feels that he is spending much of his time waiting for the typewriter 
to print out text that he saw printed several times earlier in the analysis. 

He needs to have this text to refer to, but he finds that a large percentage 
of his time is spent watching the typewriter. If this problem could be solved, 
the exploration described above could probably be done in 15 rather than 30 
minutes. The resulting decrease in connect time charges and other economies 
could be expected to reduce the cost of above analysis to about two dollars. 

The discussion in the previous paragraph is only one part of the larger 
problem concerned with the interface of the investigator with the computer. 

The standard problem has always been that of making it easy enough for 
investigators to use the computer so that they felt this use was, in fact, 
saving them time and effort. Changing "input-output" procedures, inadequate pro- 
gram write-ups * and poorly labeled output formats are continuing computer center 
problems that can be solved and often are solved — but not always. Given adequate 
interest and time, computer programmers can produce output formats that are easy to 
read and* indeed, using the kind of annotation exhibited in the paper, they 
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can, and have, produced useful sample problems. Unfortunately, this kind of 
work is seldom given high priority and, hence, is seldom done. 

Another continuing problem of statistical computing is that of turn- 
around time which can range from 15 minutes to a day or more depending on the 
demands being placed on a particular computer. At a computer center that 
is efficiently run and not overtaxed, an investigator may be able to get 
five or six runs per day either by physically carrying his input to the 
computer center or submitting his jobs through remote batch-job-entry. 

This, however, is inefficient. Each batch submission takes time to prepare 
and deliver, and this costs money. (Our accounting practices do not place 
any charge on student time misspent, but the costs are real.) In a work 
environment, accounting practices are more realistic. In the example 
we have presented, a minimum of six runs would be required to get out the 
full Bayesian analysis. Assuming no mishaps (an event having probability 
much less than one under ideal conditions) , one could hardly expect to get 

this work done in less than a day. Indeed, one would not do this by computer. 

It would be too costly. 

A skilled worker could do the job in a couple of hours, but that extra 

hour and forty-five minutes of work is a poor alternative to the two dollars 

of projected computer cost. More importantly, what of the investigator who 
has not maintained current computer skill? Such a person would probably spend 
a day on the project, with or without the computer and then, with high 
probability, not get it right. 

For the reasons given above and for others not touched upon here, it is 
certain that interactive computing will be the standard computing mode in the 
not too distant future. Given the acceptance of this conclusion, it then becomes 
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clear that a beginning should be made in the study of the inter face between 
man and computer in an interactive environment . 

An admitted difficulty of this study is that hardware technology is 
changing so swiftly that by the time any evaluation and development is 
complete, it will be less than completely up to date. Nevertheless, certain 
projections concerning hardware development and costs can be made with some 
confidence, and one of these will be touched on shortly. 

The basic problem of giving the investigator the information and 
guidance he needs without continually reprinting a book is easily solved 

by the use of a cathode ray tube terminal. With at least one currently 
available terminal, printing can be done at 240 characters per second, 
as compared to 15 characters per second for a typewriter terminal. Second, 
basic instructions and guidelines can remain on the screen (in half tone if 
desired) with only investigator responses and computations therefrom changing. 

Thus, investigators can read or ignore background information according to 
their needs. 

At present, cathode ray tube terminals cost about $3,000 each. Within 
three years improved versions of the same terminals will be available in lots 

i 

of three or more at half that price. Either of these figures is less than the cost 
of purchasing additional typewriter terminals. There are other equipment 
and personnel costs involved in interfacing the terminals to a large computer. 
These are not small but are largely of a one-time nature. One approach involves 

I 

the use of a mini-computer to accomplish the interface. 

Teaching and use of statistics in education has changed little in the 

last 15 years. The computer has made computational work easier and has made 
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it possible for more and more sophisticated work to be done. Computer 
training, however, has always been separate from statistics training under 
the assumption that when the Ph.D. candidate got to his dissertation, he 
could learn what he needed to know about programming to get his thesis done. 

What is being proposed here is that interaction with the computer be 
phased into the teaching of statistics at all but, perhaps, the most elementary 
level. Thus, almost from the beginning, the student will learn to use the 
computer in an efficient and effective manner, and he will be learning to 
use the computer in precisely the way that he will use it on the job. 

The interactive program exhibited here was developed over the last 
year, largely through a trial and error process. Refinements of the text 
and changes in the programming are introduced whenever a meaningful improvement 

seems possible. The example presented in this paper was prepared for 
presentation at a seminar sponsored by The International Statistical Institute 
and held at The Health Sciences Computing Facility in Los Angeles, August 27- 
September 1, 1971. I am Indebted to the organizers and participants of this 
conference both for specific comments and for general encouragement. 
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